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PREFACE 


This  working  note  is  part  of  a  study  conducted  for  the  Office  of  the 
Assistant  Secretary  of  Defense  (Manpower,  Reserve  Affairs,  and  Logistics). 
Its  objective  is  to  assess  the  feasibility  of  alternative  maintenance  support 
concepts  for  Army  electronic  equipment  that  would  reduce  the  requirement  for 
highly  skilled  technicians. 

A  previous  report  (1)  documented  an  "analytic  baseline"  for  the  air- 
defense-peculiar  electronics  community.  It  showed  that  the  required  niimbers 
of  air  defense  electronics  maintenance  personnel  would  decrease  in  the  1980s 
(without  changing  the  Army's  planned  support  concepts).  However,  the  skills 
required  to  maintain  the  new  weapon  systems  were  found  to  be  well  beyond  those 
required  for  the  currently  fielded  systems.  Additionally,  it  was  shown  that 
the  Army  is  hard  pressed  to  meet  even  modest  peacetime  readiness  requirements 
for  the  current  systems  due  to  a  variety  of  factors.  One  of  them  is  a  lack  of 
maintenance  skills,  but  the  singular  impact  of  this  factor  on  operational 
availability  could  not  be  quantified. 

This  note  explores  the  issue  of  maintenance  complexity  in  more  detail. 
The  objective  is  to  arrive  at  quantitative  measures  of  maintenance  task/job 
complexity.  Research  on  this  topic  has  been  very  limited  to  date.  The  pro¬ 
posed  approach,  entailing  a  taxonomy  of  criterion  measures  and  predictors,  is 
thus  a  first  attempt  at  structuring  a  methodology  for  eliciting  quantitative 
indicators  of  maintenance  complexity.  Further  research  will  be  required  to 
develop  the  appropriate  scales  and  relative  weights  for  synthesizing  the 
indicators  into  composite  measures  or  scores  which  correlate  with  the 
criterion  measures  of  maintenance  complexity. 


Once  the  proposed  approach  has  been  validated,  its  potential  applications 
are  twofold.  First,  the  taxonomy  may  be  profitably  applied  during  the 
acquisition  process  to  estimate  future  changes  in  job  complexity  for  specific 
military  occupational  specialties  supporting  the  new  system.  Quantification 
of  these  changes  could  highlight  their  repercussions  on  required  skill  levels, 
job  design,  job  aids,  training  requirements  and  personnel  selection.  Second, 
the  taxonomy  could  be  applied  in  a  macro  sense  to  obtain  trend  data  on  the 
composition  of  the  required  maintenance  skills  in  each  of  the  Services.  Such 
trend  data  might  provide  useful  input  into  defense  manpower  studies  addressing 
manpower  supply,  retention,  and  training  problems. 

In  preparing  this  report,  the  authors  benefited  from  the  experience  of 
numerous  individuals.  The  contributions  by  Joseph  Wohl  (MITRE  Corporation}, 
John  Folley  (Applied  Science  Associates,  Inc.),  Edwin  Fleishman  (Advanced 
Research  Resources  Organization),  Douglas  Towne  (University  of  Southern 
California),  and  Gerard  Deignan  (Air  Force  Human  Resources  Laboratory)  are 
particularly  acknowledged. 
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1.  MAINTENANCE  COMPLEXITY 


MANIFESTATIONS  OF  COMPLEXITY 

Maintenance  complexity  is  more  easily  described  in  terms  of  its  mani¬ 
festations  than  defined  in  a  few  words.  Figure  1-1  illustrates  how 
maintenance  complexity  can  manifest  itself  in  the  support  of  a  weapon  system. 
This  example  applies  to  the  Improved  HAWK  missile  system,  but  it  is 
representative  of  problems  the  Services  are  encountering  in  supporting  complex 
electronics  systems. 

What  happens,  is  that  a  number  of  factors  interact,  resulting  in  reduced 
availability  of  the  system,  over-consumption  of  spares,  and  a  significant 
increase  in  the  maintenance  burden.  These  factors  include  system  complexity, 
built-in  test  equipment  (BITE)  which  does  not  perform  as  expected,  technicians 
incapable  of  troubleshooting,  and  automated  test  equipment  at  the  direct 
support  (DS)  maintenance  shop  which  cannot  fully  simulate  the  tactical  end 
item  within  the  same  tolerances.  The  direct  consequences  are;  multiple 
removals  of  line  replaceable  units  (LRUs)  to  correct  singular  malfunctions;  a 
resulting  high  no  evidence  of  failure  (NEOF)  rate  for  the  LRUs  sent  to  the  DS 
shop;  an  unplanned  increase  in  the  maintenance  burden  of  the  DS  shop,  in  this 
case  amounting  to  25%  of  the  productive  man-hours;  a  high  proportion  of  LRUs 
returned  to  the  organizational  maintenance  level  failing  the  system  check 
(FSC)  and  cycled  back  again  to  the  DS  shop  (1). 

Maintenance  complexity  also  manifests  itself  in  the  shape  of  the 
frequency  distributions  of  actual  repair  times.  Wohl  has  performed  some 
interesting  research  along  this  line  (2).  Figure  1-2  is  adapted  from  his 
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ORGANIZATIONAL  SUPPORT  LEVEL 


IMPAGT:  •  Increased  Supply  Demand 

•  Increased  Maintenance  Workload 


FIGURE  1-2.  CUMULATIVE  FREQUENCV  DtSTRtBUTlQN  (CFD)  OF  REPAIR  TIME  FOR  EQUIPMENT  I) 

PLOTTED  ON  WEIBUl.L  PROBABILITY  PAPER 


Source:  "DLaguoatic  Beli.ivior,  System  Ct>ro|>]  ex  i  ty ,  and  Repair  Time:  A  Predictive  Theory, 

hy  dosepli  C.  Wohl,  MITKK/Bedford  M80-00008,  April  1980.  Reprinted  by 
perm  is.s  ion. 
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research  and  illustrates  a  basic  phenomenon  of  fault  isolation.  The  cumula¬ 
tive  frequency  distributions  of  actual  repair  times  for  a  mix  of  electronics 
equipments  plotted  on  the  Weibull  probability  scale  have  a  characteristic 
shape  showing  the  existence  of  two  distinct  fault  isolation  processes.  The 
majority  of  repairs  (65%  to  80%  in  Wohl  s  sample  consisting  of  equipments 
first  fielded  in  the  late  1960s)  are  completed  within  a  short  time  (about  two 
hours),  thanks  to  traditional  methods  of  maintainability  design  (BITE,  self¬ 
test  and  self-diagnostics,  troubleshooting  aids,  modularization,  and  equipment 
packaging).  The  remaining  repairs  require  a  much  longer  time  to  complete  due 
to  diagnosis  and  isolation  difficulties;  i.e.,  some  of  the  system's  malfunc¬ 
tions  "escape"  its  maintainability  features. 

Importantly,  as  Wohl  points  out,  the  low  slope  (p  <  1.0)  of  this  portion 
of  the  curve  indicates  a  decelerating  process  in  which  the  probability  of 
fault  isolation  during  a  given  interval  decreases  with  time,  suggesting  in¬ 
creasing  rarity  of  failure  mode,  increasing  difficulty  in  symptom  interpreta¬ 
tion,  or  increasing  circuitry  complexity.  (Wohl's  analysis  is  based  on 
validated  maintenance  data  from  a  special  data  collection  effort.  The  data 
consist  of  active  repair  times  by  well-motivated  military  maintenance 
personnel.)  Clearly,  the  specific  location  of  the  "breakpoint"  in  this  graph 
for  a  given  equipment  is  an  indicator  of  maintenance  complexity  manifesting 
itself  in  a  long  tail  of  the  repair  time  distribution.^ 

MAINTENANCE  COMPLEXITY  VERSUS  MAINTAINABILITY 

NEOF  rate,  FSC  rate,  and  tail  of  repair  time  distribution  are  the  key 
manifestations  of  maintenance  complexity.  For  our  purposes,  maintenance 
complexity  is  defined  as  the  product  of  all  man-machine  interface  factors 

^Wohl  notes  that  he  has  found  the  same  phenomenon  for  almost  all  complex 
electronics  under  field  conditions,  with  the  sole  exception  of  flight-line 
maintenance  of  highly  modularized  avionics  for  which  (3  =  1.0  (i.e., 
exponentially  distributed). 
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which  influence  the  effectiveness  of  a  technician ' s  j  ob  performance  as 
measured  in  terms  of  NEQF  rate ,  FSC  rate ,  and  tail  of  repair  time  distribution 
(region  with  slope  less  than  1.0  in  Figure  1-2).  Thus,  included  in  our  def¬ 
inition  are  such  factors  as  equipment  design,  maintainability  design,  test 
equipment  performance  (built-in  or  off-line),  quality  of  maintenance  diag¬ 
nostics  software,  adequacy  of  job  performance  aids  (especially  for  trouble¬ 
shooting),  and  the  technician's  ability  to  do  the  job.  Unfortunately,  this 
type  of  definition  virtually  precludes  the  possibility  of  e.xpressing  mainte¬ 
nance  complexity  in  terms  of  a  simple  mathematical  relationship  of  the  under¬ 
lying  factors.  Instead,  complexity  will  have  to  be  expressed  in  terms  of 
multiple  scales;  i.e.,  a  taxonomy  or  checklist. 

Quantitative  assessment  of  maintenance  complexity  is  important,  not  as  a 
solution  to  current  maintenance  support  problems,  but  to  give  the  issue  more 
visibility  in  the  acquisition  process.  The  inherent  complexity  of  a  piece  of 
equipment  is  normally  determined  by  the  operational  requirements  of  the  weapon 
system.  In  certain  mission  areas,  the  enemy  may  have  a  quantitative  superior¬ 
ity,  and  U.S.  policy  has  been  to  compensate  by  ensuring  qualitative 
superiority.  The  result  is  increasingly  sophisticated  and  complex  equipments. 
Exploitation  of  the  built-in  operational  capabilities  of  those  equipments 
depends  on  the  skills  and  training  of  those  operating  and  maintaining  them. 

The  objective  of  focusing  on  maintenance  complexity  in  the  acquisition 
process  would  be  to  ascertain  that  equipment  complexity  is  adequately  compen¬ 
sated  for  by  (1)  a  systematic  allocation  of  diagnosis  functions  between  BITE 
and  technician,  (2)  effective  troubleshooting  aids  for  the  technician,  (3)  a 
job  (MOS)  design  which  does  not  require  the  technician  to  perform  an  impos¬ 
sible  variety  of  tasks  across  different  technologies  and  subspecialties,  (4'  a 
trainer  guarantee  of  certain  personnel  aptitude  and  training  stanuards,  and 


(5)  a  user  guarantee  of  certain  managed  on  the  job  training  standards.  What 
is  clearly  needed  is  a  more  sophisticated  scrutiny  of  maintainability  than 
afforded  by  just  focusing  on  mean  time  to  repair.  Otherwise,  current  mainte¬ 
nance  support  problems  are  bound  to  grow  worse  with  the  influx  of  more  complex 
electronics  systems  in  the  1980s.  While  in  peacetime  any  maintenance  capabil¬ 
ity  shortcomings  may  be  compensated  through  the  supply  system  (increased 
stockage  or  pipelines  of  spares,  intensive  management  of  supply,  use  of  readi¬ 
ness  floats),  wartime  sustainability  depends  on  effective  use  of  available 
spares  and/or  cannibalization  which  requires  maintenance  skill  proficiency 
(36) . 

■APPROACH  TO  QUANTITATIVE  .ANALYSIS 

The  proposed  approach  entails  development  of  a  taxonomy  of  maintenance 
complexity  indicators.  Following  review  and  acceptance  of  this  taxonomy,  the 
second  step  would  be  to  develop  specific  scales  and  weights  to  permit  indi¬ 
cators  to  be  compared  between  weapon  systems  as  well  as  combined  into  com¬ 
posite  measures.  Following  validation  of  these  data,  the  third  and  final  step 
would  be  to  institutionalize  evaluation  of  maintenance  complexity  in  the 
system  review  process. 

On  a  more  near-term  basis,  an  abbreviated  version  of  the  taxonomy  could 
be  validated  and  applied  on  a  macro  level.  The  objective  of  this  exercise 
would  be  to  gain  a  more  precise  understanding  of  evolving  trends  of  mainte¬ 
nance  complexity  in  a  mission  area  or  Service  aggregate,  given  the  current  and 
planned  inventories  of  weapon  systems  and  associated  support  concepts.  Such  a 
macro  analysis  of  maintenance  complexity  could  illustrate  shifts  in  skill 
requirements  and  the  resulting  repercussions  in  terms  of  skill  level  distribu¬ 
tions,  training  requirements,  and  aptitude/ability  prerequisites. 
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2.  OVERVIEW  OF  PROPOSED  TAXONOMY 


INTRODUCTION 

The  ultimate  criterion  of  maintenance  job  performance  is  the  availability 
and/or  wartime  sustainability  of  the  weapon  system.  Criterion  measures  of 
availability  are  varied  and  depend  on  the  purpose  of  the  evaluation 
(mission-capability,  maintenance  man-hours,  or  supply  requirements).  Two 
standard  measures  are  operational  availability  (.Aq)  and  inherent  availability 
(Aj),  conventionally  defined  as  follows: 

,  _  MTBF  _  MTBF 

0  MTBF  +  MDT  MTBF  +  MTTR 

where:  MTBF  =  mean  time  between  failures 
MDT  =  mean  downtime 

=  .MTTR  +  MLDT  +  MADT 
MTTR  =  mean  time  to  repair 

MADT  =  mean  administrative  delay  time  (i.e.,  waiting  for 
maintenance  personnel  with  requisite  skills  or  test 
equipment) 

MLDT  =  mean  logistic  delay  time  (i.e.,  waiting  for  parts) 

A  distinction  must  be  made  between  predicted  and  actual  values  of  these 
measures.  Aq  should  be  based  on  empirical  field  data  to  have  practical 
meaning.  Aj  is  more  a  design  measure;  it  assumes  instant  availability  of  the 
requisite  maintenance  personnel,  test  equipment  and  spares,  and  is  based  on 
predicted  MTTR  defined  as  the  "mean  value  of  the  probability  distribution  of 
times  to  complete  active  corrective  maintenance  over  all  predictable 
unscheduled  actions  weighted  by  the  relative  frequencies  of  occurrence"  (28). 
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MTTR  prediction  is  typically  based  on  the  following  assumptions:  fail¬ 
ures  occur  at  predicted  rates;  only  single,  "hard"  (catastrophic)  failures 
occurring  randomly  are  included;  maintenance  is  performed  in  accordance  with 
established  procedures  by  technicians  with  appropriate  skills  and  training; 
only  active  maintenance  time  is  counted,  excluding  administrative/logistic 
delays,  fault  detection,  and  clean-up  time.  As  a  result,  actual  MTTRs  deter¬ 
mined  from  field  data  are  invariably  larger  than  predicted  MTTRs  using 
standard  MIL-HDBK-472  or  related  (25,  26)  techniques. 

For  example,  Phaller  (29)  reports  differences  by  a  factor  of  2.5  to  5.8 
for  a  mix  of  Air  Force  equipments,  with  80%  of  the  difference  due  to  opera¬ 
tional  factors  beyond  the  purview  of  MIL-HDBK-472  prediction  techniques.  (The 
most  significant  factors  identified  by  the  study  include:  preparation  and 
clean-up  times;  unavailability  of  tools,  test  equipment  or  spares;  cannot 
duplicate  rate;  training  shortcomings;  and  differences  between  actual  vs. 
predicted  failures).  Nevertheless,  a  substantial  increase  in  actual  MTTR  from 
the  predicted  value  may  have  little  impact  on  equipment  availability  if  the 
reliability  (MTBF)  is  high. 

Maintenance  complexity,  by  our  definition,  affects  repair  time  and 
quality  of  repair.  Four  criteria  are  proposed  to  measure  the  impact  of  main¬ 
tenance  complexity:  NEOF  rate;  FSC  rate;  percent  failures  not  included  in  the 
high  slope  region  (p  ^  1.0)  of  the  cumulative  frequency  distribution  (CFD)  of 
repair  times  (Figure  1-2);  and  the  complement  of  the  slope  in  the  low  slope 
(p  <  1.0)  region  of  the  CFD.^ 

^The  Air  Force  terminology  for  NEOF  is  Retest  OK(RTOK).  .Another 
phenomenon  encountered  in  the  Air  Force  is  the  inability  to  duplicate  on  the 
ground  a  failure  detected  while  the  aircraft  is  in  operation.  This  cannot 
duplicate  (CND)  rate  is  an  additional  candidate  criterion  measure  of  mainte¬ 
nance  complexity.  This  phenomenon  also  occurs  in  complex  ground  equipment  but 
is  not  usually  recorded. 
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An  increase  in  maintenance  complexity  results  in  an  increase  in  any  or 
all  of  these  four  measures,  everything  else  being  equal.  The  impact  on  is 
through  an  increase  in  MDT:  MTTR  increases  due  to  a  longer  tail  of  the  fre¬ 
quency  distribution  of  repair  times;  MLDT  increases  due  to  increased  parts 
consumption  (unless  the  effect  of  NEOF  and  FSC  is  compensated  for  by  increased 
stockage  and  pipeline  supply) ;  MADT  may  increase  due  to  increased  occurrences 
where  special  skills  are  required  (unless  this  effect  of  complexity  is  com¬ 
pensated  for  by  upgrading  the  capabilities  of  maintenance  personnel) . 

The  precise  impact  of  maintenance  complexity  on  A^  cannot  be  assessed 
apart  from  a  variety  of  operational  and  logistics  factors.  Specifically, 
maintenance  complexity  does  not  determine,  by  itself ,  the  level  of  effective¬ 
ness  of  maintenance  performance  as  measured  by  the  above  four  criterion 
measures .  One  has  to  be  careful  when  equating  a  high  NEOF  rate  (observed  from 
field  maintenance  data)  with  a  high  degree  of  maintenance  complexity.  The 
real  cause  might  well  be  unavailability  of  tools  or  test  equipment,  or  any  one 
of  a  number  of  environmental  factors  beyond  the  domain  of  maintenance 
complexity . 

Thus,  maintenance  complexity  must  be  measured  in  its  own  terms,  separate 
from,  but  predictive  of,  the  four  criterion  measures. 

STRUCTURE 

The  man-machine  interface  factors  determining  maintenance  complexity  may 
be  classified  in  terms  of  three  categories  as  shown  in  Figure  2-1:  factors 
which  are  equipment  related  (inherent  complexity);  factors  associated  with  the 
job  design,  including  the  maintenance  concept  (job  complexity);  and  factors 
expressing  the  gap  between  job  performance  requirements  and  the  technician's 
capabilities  (apparent  complexity). 


FIGURE  2-1.  FRAMEWORK  FOR  ANALYSIS  OF  MAINTENANCE  COMPLEXITY 


Inherent  Complexity 

This  component  of  maintenance  complexity  contains  the  factors  which 
identify  equipment  complexity,  including  the  design  characteristics  for  main¬ 
tainability.  Some  of  the  factors  are  known  once  the  failure  modes  and  effects 
analysis  (FMEA)  has  been  completed,  diagnosis  tasks  have  been  allocated  be¬ 
tween  man  and  machine,  and  BITE/maintenance  diagnostic  software  (MBS)  spec¬ 
ifications  have  been  determined.  Other  factors  remain  unknown  until  the 
system  has  been  fielded,  because  neither  the  FMEA  nor  maintenance  demonstra¬ 
tions  can  accurately  simulate  operational  conditions.  That  is,  validation  of 
the  BITE  and  MBS  prior  to  fielding  of  the  weapon  system  is  currently  not 
possible . 

Job  Complexity 

This  component  of  maintenance  complexity  contains  the  factors 
identifying  the  complexity  of  the  job  which  the  technician  is  required  to 
perform.  These  data  are  not  normally  under  control  of  the  program  manager. 
In  addition  to  the  types  of  data  commonly  provided  in  the  qualitative  and 
quantitative  personnel  requirements  information  (QQPRI)  or  similar  manning 
estimate  documents  based  on  the  logistic  support  analysis  (ISA),  this  com¬ 
ponent  includes  factors  pertaining  to  other  aspects  of  the  technician's  job: 
other  end  items  to  be  supported,  quality  of  job  performance  aids,  etc.  In 
other  words,  assessing  job  complexity's  contribution  to  maintenance  complexity 
in  the  Army  will  require  an  early  tentative  MOS  decision  and  involvement  of 
TRABOC  and  MILPERCEN. 

Apparent  Complexity 


This  component  of  maintenance  complexity  contains  the  factors  which 
cause  the  actual  job  to  be  more  complex  for  an  individual  than  planned. 
Potential  causes  include  improper  selection/classification  procedures,  a  poor 


approach  to  training,  absence  of  managed  OJT  in  the  field,  no  standards  of 
performance,  and  improper  personnel  utilization  (lack  of  practice). 

Difficulty  vs.  Complexity 

The  distinction  between  difficulty  and  complexity  seems  moot,  but  is 
a  basic  one.  Difficulty  refers  to  physical  attributes  such  as  lifting  a 
200-pound  LRU,  working  in  a  cramped  space,  or  probing  connections  under  ad¬ 
verse  conditions  (darkness,  rain,  freezing  or  tropical  temperatures).  Task 
difficulty  probably  affects  time  to  repair  in  the  sense  that  a  difficult  task 
may  take  longer  to  complete  than  a  cijriple  task.  In  contrast,  task  complexity 
refers  to  some  type  of  mental  requirement.  In  the  behavioral  sciences,  com¬ 
plexity  is  normally  treated  as  subordinate  to  difficulty:  a  complex  task  is 
normally  difficult,  but  a  difficult  task  is  not  complex  per  se.  In  Figure 
2-1,  we  elect  to  treat  difficulty  and  complexity  as  separate  components  of 
maintainability  to  emphasize  the  distinction  between  the  two  for  the  purposes 
of  this  study.  Difficulty  may  affect  MTTR  but  should  not  exhibit  itself  in  a 
task  time  distribution  characteristic  of  complexity  (Figure  1-2). 

SVALUATION  MEASURES 

The  intended  uses  of  the  taxonomy  are  to  compare  the  complexity  of  dif¬ 
ferent  weapon  systems  and  associated  maintenance  jobs  (micro  analysis  for 
DSARC  purposes)  or  to  illustrate  overall  trends  in  maintenance  job  complexity 
and  maintenance  capabilities  (macro  analysis  for  manpower,  personnel,  and 
training  program  purposes).  These  intended  uses  suggest  the  need  for  an  over¬ 
all  measure  of  maintenance  complexity  predictive  of  the  four  criterion  meas¬ 
ures.  While  an  anchored,  benchmark  scale  could  be  used  to  measure  overall 
maintenance  complexity,  such  a  single  measure,  however,  would  not  provide  in¬ 
sight  into  what  causes  complexity.  Thus,  we  propose  that  weapon  systems  be 
compared  in  terms  of  inherent  complexity;  maintenance  jobs  be  compared  in 
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terms  of  job  complexity;  and  maintenance  job  performance  be  compared  in  terms 
of  apparent  complexity  (see  Figure  2-2). 

Inherent  Complexity 

Current  maintainability  measures  utilized  to  evaluate  inherent 
complexity  are  expressed  in  terms  of  MTTR  or  derivatives  thereof.  These 
measures,  however,  are  imprecise  as  they  reflect  both  the  maintainability 
characteristics  of  the  hardware  (complexity  as  well  as  difficulty)  and  the 
capabilities  of  maintenance  personnel  (including  quality  of  aids).  To  al¬ 
leviate  some  of  these  measurement  problems,  an  earlier,  unpublished  version  of 
this  document  proposed  the  following  criterion  measure:  the  ratio  of  maximum 
repair  time,  (95  per  cent),  to  MTTR  as  demonstrated  by  expert  (con¬ 

tractor)  technicians  in  a  maintenance  demonstration  (M-demo) .  Further  analy¬ 
sis,  however,  suggests  that  this  ratio  is  not  a  useful  measure  because  its 
validity  is  limited  to  conditions  where  repair  tasks  are  homogeneous  in  terms 
of  the  percent  of  repair  task  time  expended  in  fault  isolation.  In  reality, 
the  corrective  maintenance  tasks  associated  with  a  given  weapon  system  or 
equipment  show  a  wide  range  in  the  percent  of  repair  time  required  for  fault 
isolation,  so  that  the  repair  time  distribution  is  not  indicative  of  the  fault 
isolation  time  distribution.  Consequently,  interpretations  based  on  analysis 
of  repair  times  may  be  misleading. 

For  example,  conventional  engineering  rule  of  thumb  used  in  pre¬ 
liminary  design  is  to  assume  that  the  ratio  M  (95  percent) /MTTR)  is  3  (27). 

max 

If  the  overall  repair  time  is  lognorroally  distributed,  the  ratio  follows  from 
the  following  equation: 


max((>) 

MTTR 


z^Vln(l+n^) 

VT+n^ 


2-7 


FIGURE  2-2.  MEASURES  OF  MAINTENANCE  COMPLEXITY 
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where:  z-  =  table  value  for  standard  normal  distribution  (z  = 

^  1.64) 

n  =  coefficient  of  variation  =  a/MTTR 

Pliska  (28)  reports  a  wide  range  of  values  for  the  coefficient  of 
variation  based  on  formal  maintenance  demonstrations  of  various  equipments. 
(For  example,  the  largest  q  value  reported  in  that  study  is  1.63  for  display 
equipment;  the  smallest  value  is  0.42  for  data  converters;  the  weighted 
average  for  the  equipments  sampled  is  0.864.)  The  study  suggests  that  for 
prediction  purposes  in  preliminary  design,  q  values  be  based  on  these  sample 
data  in  lieu  of  the  value  of  1.07  in  MIL-HDBK-472 .  The  latter  is  based  on 
older  technology  (mid-1960s);  the  suggestion  is  that  q  decreases  as  the  degree 
of  equipment  modularity  and  fault  isolation  automaticity  increase.  Wohl's 
data  (2)  would  indicate  that  this  is  not  necessarily  true. 

An  explanation  of  this  disparity  in  statistical  interpretations  is 
provided  by  our  earlier  statement  that  repair  tasks  are  not  homogeneous,  but 
show  a  wide  variation  in  the  percent  of  task  time  spent  in  fault  isolation. 
This  is  the  reason  why  our  previously  proposed  measure  of  inherent  complexity 
must  be  rejected.  Instead,  the  proper  criterion  measure  is  an  equipment 
complexity  index  based  on  the  distribution  of  fault  isolation  times  for  all 
equipment  failures,  when  fault  isolation  is  performed  by  expert  technicians. 
It  is  recognized  that  this  index  may  be  unavailable  prior  to  fielding  of  the 
weapon  system  because  failure  modes  of  the  fielded  system  may  differ  from 
those  established  through  analysis  (FMEA)  or  encountered  in  the  laboratory 
(development  testing  and  M-demos).  Measurement  of  this  index  thus  requires 
follow-on  test  and  evaluation. 
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Job  Complexity 

Two  measures  of  job  complexity  are  proposed;  (1)  the  time  required 
to  attain  mastery  (journeyman-level  capability)  of  all  tasks  for  the  "average" 
qualified  individual  in  the  "normal"  Service  environment,  and  (2)  the  basic 
abilities  and  aptitudes  required  to  attain  mastery.  The  first  measure  is 
widely  accepted  in  the  literature,  but  it  depends  on  expert  raters--an  ap¬ 
proach  which  exhibits  certain  well  known  problems  (35). 

The  Air  Force  is  currently  engaged  in  a  major  research  program  to 
develop  a  job  complexity  index  for  all  jobs  based  on  this  first  measure  (10). 
The  approach  is  based  on  a  25-point,  anchored,  benchmark  scale  of  learning 
difficulty  for  each  of  a  representative  sample  of  tasks.  Occupational  survey 
data  (including  the  percent  of  time  specific  tasks  are  performed  in  each  job) 
are  used  to  obtain  an  overall  measure  of  difficulty  for  each  Air  Force 
specialty,  referred  to  as  the  "average  task  difficulty  per  unit  time  spent". 
Once  these  indices  have  been  obtained,  the  objective  is  to  use  them  as  an 
empirical  basis  for  aligning  aptitude  requirements  across  specialties.  A 
similar  rating  approach  would  be  necessary  in  the  Army  for  validating  the 
proposed  taxonomy. 

We  see  this  job  complexity  measure  more  as  a  measure  of  job  design 
(number  and  diversity  of  tasks  allocated  to  a  billet)  indicating  experience  or 
skill  level  requirements  than  as  a  measure  implying  aptitude  requirements.  It 
is  fair  to  say  that  the  Army  ignores  job  complexity  in  developing  manpower 
requirements  for  a  new  weapon  system:  maintenance  tasks  are  allocated  so  as 
to  minimize  the  number  of  billets  required,  and  paygrades  are  authorized  as  a 
function  of  the  number  of  billets  via  standard  grade  authorization  tables. 
For  example,  if  estimated  maintenance  man-hours  do  not  justify  more  than  one 
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or  two  billets,  a  grade  of  E-4  (perhaps  one  E-5  for  certain  MOSs)  is  typically 
authorized,  with  little  regard  to  the  individual's  ability  to  perform  the  job. 

Field  data  indicate  that  even  in  the  Navy  (which  probably  does  the 
best  job  of  classifying  and  training  maintenance  personnel)  the  average  mainte¬ 
nance  technician  (somewhere  between  E-4  and  E-5)  is  capable  of  performing  onl, 
65%  of  his  work  center's  nontroubleshooting  tasks,  with  a  much  lower  per¬ 
centage  for  troubleshooting  tasks;  and  that,  on  the  average,  3.7  years 
experience  is  required  (following  entry-level  training)  to  achieve  the  MTTR  of 
a  group  of  experienced  technicians  for  those  tasks  that  the  apprentice  is 
capable  of  performing  (30) .  The  latter  data  are  in  close  agreement  with  the 
data  reported  by  RAND  on  job  performance  for  a  mix  of  occupational  specialties 
(33) .  Electronics  troubleshooting  proficiency  cannot  be  expected  from  tech¬ 
nicians  with  less  than  nine  years  experience.  The  typical  E-5  with  over  four 
years  experience  can  only  fault  isolate  two  out  of  three  failures  (24) . 

Some  of  these  job  performance  problems  are,  of  course,  independent 
of  job  complexity  and  will  be  returned  to  in  the  next  section  on  apparent  com¬ 
plexity.  In  any  case,  a  job  complexity  measure  as  proposed  is  clearly  related 
to  the  ultimate  criterion  of  the  taxonomy,  effectiveness  of  job  performance. 
If  institutionalized  in  the  Services,  this  measure  would  also  lend  itself  for 
remedial  action  in  the  job  design  and  grade  authorization  process. 

The  second  and  more  fundamental  measure  proposed  for  job  complexity 
consists  of  the  basic  abilities  and  aptitudes  required  to  attain  proficiency. 
At  present,  no  satisfactory  measure  is  readily  available  for  this  purpose. 
The  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  is  used  to  screen  can¬ 
didates,  based  on  their  Armed  Forces  Qualification  Test  composite  score,  and 
to  determine  their  qualifications  for  assignment  to  specific  occupational  spe¬ 
cialties,  based  on  aptitude  composites.  Area  aptitude  composites  differ  among 
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the  Services;  the  Air  Force  uses  4,  the  Marine  Corps  7  ,  the  Army  10,  and  the 
Navy  11. 

Both  the  ASVAB  subtests  and  the  aptitude  composites  have  changed 
from  time  to  time.  For  example,  with  the  introduction  of  the  new  ASVAB  (Forms 
8/9/10,  October  1980)  the  electronics  aptitude  (EL)  composite  (which  happens 
to  be  the  same  for  all  four  Services)  comprises  general  sciences,  arithmetic 
reasoning,  mathematics  knowledge,  and  electronics  information.  The  previous 
ASVAB  version  used  an  EL  composed  of  arithmetic  reasoning  and  electronics 
information . 

All  Army  electronics  maintenance  .MOSs  use  a  cutting  score  on  the  EL 
as  the  single  selection  standard  (prerequisite).  There  is  no  scientific 
method  for  determining  the  "right"  aptitude  prerequisite  for  an  .MOS .  Cutting 
scores  have  been  changed  in  the  Army  many  times  without  a  change  ui  MOS  skill 
requirements.  Such  changes  are  made  on  the  basis  of  academic  attrition  ex¬ 
perience  in  entry-level  school  training  and  actual  or  expected  difficulty  in 
meeting  training  quotas. 

Empirical  data  show  that  EL  is  predictive  of  academic  success; 
trainees  with  a  higher  EL  score  have  a  lower  probability  of  academic  at¬ 
trition.  In  those  cases  where  different  services  have  identical  equipment  and 
job  descriptions,  selection  standards  typically  differ.  For  example,  for 
Improved  HAWK  maintenance  personnel,  the  Army  currently  requires  an  EL  of  105 
for  organizational  maintenance  MOSs  and  an  EL  of  100  for  direct  support  main¬ 
tenance  MOSs.  (The  .Army  uses  standard  scores  on  a  scale  from  40  to  160.)  For 
the  same  jobs,  the  .Marine  Corps  requires  a  GT  of  110  and  an  EL  of  120. 

A  great  deal  has  been  written  about  the  ASVAB,  but  we  have  not  been 
able  to  locate  any  data  showing  a  relationship  between  aptitude  and  job  per¬ 
formance  (other  than  written  skill  tests).  Instead,  much  of  the  research 
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published  over  the  Last  15  years  shows  that  lower  aptitude  personnel,  given 
different  training  and  job  aids,  are  capable  of  better  maintenance  job  per¬ 
formance  than  higher  aptitude  personnel  trained  conventionally  with  conven¬ 
tional  aids  (8).  IvTiile  this  does  not  disprove  that  high  aptitude  personnel 
may  indeed  perform  better,  the  data  suggest  that  caution  is  warranted  in  using 
MOS  selection  standards  as  a  proxy  measure  of  job  complexity. 

Aptitude  tests  fail  to  predict  job  performance  because  they  do  not 
tap  the  basic  abilities  and  personal  traits  which  are  predictive  of  job  per¬ 
formance.  Some  of  the  most  useful  research  in  this  area  has  been  performed  by 
Fleishman  and  his  coworkers  (9).  This  research  demonstrates  that  effective 
performance  of  a  complex,  cognitive  task  such  as  electronics  troubleshooting 
depends  on  a  number  of  cognitive  and  psychomotor  abilities  which  can  be  meas¬ 
ured  and  tested.  These  basic  abilities  are  permanent  individual  attributes. 
The  implication  is  that  persons  without  these  abilities  cannot  be  expected  to 
become  proficient  at  certain  jobs  regardless  of  a  high  aptitude  score,  unless 
the  tasks  are  fully  procedurized  in  a  job  aid  which  is  100%  accurate. 

In  summary,  given  the  current  absence  of  valid  measures  of  both 
learning  difficulty  and  aptitude/ability  requirements,  there  seems  little 
choice  but  to  use  an  anchored,  benchmark  scale  to  measure  job  complexity--a 
scale  which  must  be  developed  by  experts. 


.Apparent  Complexitv 


This  component  of  maintenance  complexity  is  probably  peculiar  to  the 


Services.  It  refers  to  a  second-order  effect  resulting  in  an  increase  in  job 


complexity  as  perceived  by  the  person  doing  it.  What  this  component 
represents  is  the  gap  between  job  requirements  (abilities,  skills,  knowledge, 
experience)  and  the  individual's  capabilities.  Some  of  the  factors 
involved  --  job  design,  pay-grade  authorization,  classification  --have  already 


been  mentioned.  .Another  factor  is  the  quality  of  training  maintenance  person¬ 
nel  receive.  .An  obvious  measure  of  this  is  the  percentage  of  tasks  the  tech¬ 
nician  can  perform  satisfactorily.  When  the  Army's  Skill  Qualification  Test 
has  become  a  more  reliable  and  valid  measure  of  job  performance,  its  results 
will  provide  a  readily  available  measure  of  apparent  complexity. 

The  DoD-wide  "rationalization"  of  technical  skill  training  in  the 
mid-1970s  under  the  Instruction  System  Development  program  may  have  been  a 
good  concept,  but  its  implementation  has  been  disappointing.  One  reason  for 
this  may  be  that  resource  constraints  have  limited  the  quality  of  the  required 
front-end  analysis.  -As  a  result,  individual  skill  training  still  does  not 
meet  requirements  (11).  A  more  important  reason  is  the  absence  of  an  ef¬ 
fective  managed-on-the-job  training  (MOJT)  program  for  any  of  the  Army's 
electronics  maintenance  MOSs  we  have  examined. 

Once  again,  resources  are  the  issue.  No  MOJT  billets  (.instructors 
nor  trainees)  are  authorized  in  the  .Army's  force  structure.  There  is  a  short¬ 
age  of  NCOs  possessing  both  technical  and  instructor  skills.  Lack  of  mainte¬ 
nance  training  simulators  minimizes  the  extent  of  troubleshooting  taught  at 
TRADOC  schools.  Overseas,  readiness  is  perceived  as  more  important  than 
maintenance  training  so  that  time  and  equipment  for  hands-on  training  are 
limited.  In  FORSCOM  also,  maintenance  training  has  to  compete  with  many  other 
operational  requirements.  And  excessive  personnel  turbulence  makes  any  MOJT 
program  difficult  to  implement.  Even  in  the  Air  Force,  which  does  have  an 
.MOJT  program  conducted  by  Field  Training  Detachments,  there  are  serious  prob¬ 
lems  with  the  quality  of  OJT  (3).  Clearly,  budgetary  considerations  rather 
than  requirements  have  affected  the  level  of  skill  training. 

To  counteract  this  problem  (as  well  as  to  open  these  MOSs  to  lower- 
aptitude  personnel)  much  research  over  the  past  decade  has  been  devoted  to  job 


2-14 


or  skill  performance  aids  (JPAs  or  SPAs)  (7,  8,  14,  17  -  23).  This  research 
has  focused  primarily  on  specific  formats  for  troubleshooting  and  nontrouble¬ 
shooting  JPAs  and  experiments  demonstrating  potentially  large  savings  from 
implementing  JPAs.  The  experiments  conducted  in  the  Services  show  the  follow¬ 
ing  payoffs; 

training  time  reduced  by  60%  for  troubleshooting  tasks  and  up  to 
100%  for  nontroubleshooting  tasks 

active  repair  time  (excluding  fault  diagnosis)  reduced  by  up  to 
33% 

fault  diagnosis  time  reduced  by  up  to  67% 
reduction  in  false  removals  (NEOF)  by  15% 

Westinghouse  reports  that  its  experience  with  JPAs,  based  on  both  Air  Force 
and  industry  standards,  shows  the  following  potential  savings:  MTTR  reduced 
by  40%  (including  10%  due  to  less  look-up  time  compared  with  conventional 
technical  manuals),  error  rate  reduced  by  75%,  spares  consumption  reduced  by 
30%,  and  training  time  reduced  by  25%  (18). 

To  date,  implementation  of  .JPAs  has  been  piecemeal,  and  the  promised 
payoffs  have  not  been  realized.  The  Services,  understandably,  are  reluctant 
to  implement  JPAs  on  a  scale  that  might  ensure  that  the  potential  benefits  are 
reaped.  Some  practical  problems  related  to  job  structure  and  personnel  man¬ 
agement  are  pointed  out  by  Malehorn  (18).  Another  issue  is  that  JPAs  must  be 
accurate;  i.e.,  contractor  support  might  be  required  in  the  first  couple  of 
years  of  a  newly  fielded  system  until  most  of  the  engineering  changes  have 
been  fielded  and  incorporated  into  the  JPAs.  Also  fully  procedurized  aids  for 
digital  troubleshooting  have  been  unsuccessful  to  date,  although  a  new  ap¬ 
proach  (14)  may  overcome  this  shortcoming.  Yet  another  problem  is  that  ade¬ 
quate  JP.As  are  significantly  more  expensive  than  traditional  manuals.  As  long 
as  program  managers  are  driven  by  design-to-unit-production  cost  rather  than 
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life  cycle  cost,  there  is  a  disincentive  to  spend  much  money  on  the  high- 
quality  front-end  analysis  required  for  proper  JPAs .  Folley  (18)  points  out 
that  research  should  be  reoriented  towards  the  program  manager's  point  of 
view,  addressing  the  question  of  the  most  cost-effective  mix  of  aiding/ 
training.  Bond  and  Towne  (6)  provide  an  excellent  overview  of  the  multi¬ 
disciplinary  approaches  required  to  solve  the  troubleshooting  problem.  They 
insist  that  even  the  most  effective  aids  will  not  permit  finding  all  failures. 
Thus,  the  services  would  always  need  expert  troubleshooters  in  addition  to 
JPAs  for  lower  skilled  personnel. 

At  present,  a  number  of  pilot  programs  are  underway  to  assess  the 
potential  of  JP.As  in  the  context  of  multi-level  training:  minimum  entry- 
level,  task-oriented  training  combined  with  a  fully  funded  JPA  program,  and 
advanced  technical  training  for  those  first-termers  who  reenlist.  If  these 
programs  are  successful  and  if  JPAs  are  fully  institutionalized  in  the  Serv¬ 
ices,  job  complexity  could  be  significantly  reduced,  and  apparent  complexity 
minimized.  Until  then,  apparent  complexity  will  contribute  substantially  to 
maintenance  complexity  and  cause  ineffective  job  performance. 
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3.  DETAILED  SPECIFICATION  OF  TAXONOMY 


INTRODUCTION 

Figure  3-1  provides  an  overview  of  the  maintenance  complexity  taxonomy. 
Each  of  the  three  components,  inherent,  job,  and  apparent  complexity,  is  made 
up  of  factors  which  are  in  turn  measured  by  indicators.  Indicator  values  pre¬ 
dict  component  measures,  but  the  specific  statistical  relations  are  unknown. 
We  have  not  developed  the  necessary  scales  and  weights  for  the  indicators  to 
obtain  statistically  valid  measures.  Many  of  the  indicators  should  probal)ly 
be  measured  as  indices  on  a  nonlinear  scale. 

The  discussion  in  this  chapter  is  focused  on  complexity  at  the  system 
level  and  on-equipment  maintenance  jobs.  Some  modifications  would  be  required 
to  specify  inherent  complexity  for  off-equipment  maintenance,  though  job  and 
apparent  complexity  specifications  would  remain  the  same. 

INHERENT  COMPLEXITY 

Overall  .Measure 

The  overall  measure  of  equipment  (hardware  and  software)  inherent 
complexity  is  the  equipment  complexity  index  representing  the  distribution  of 
fault  isolation  times  for  all  equipment  failures  when  fault  isolation  is 
performed  by  expert  technicians.  Obtaining  this  measure  will  require  changes 
in  maintenance  data  collection  formats. 

Factors  and  Indicators 

The  factors  determining  inherent  complexity  are: 
functional  design 
failure  modes 
self-test  characteristics 
diagnostic  requirements 
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interconnect ivity 


weapon  system  breakdown  structure 

Table  3-1  shows  the  indicators  associated  with  these  factors.  Each  is  ex¬ 
plained  briefly  below.  The  specific  factors  to  be  used  for  assessing  inherent 
complexity  depend  on  where  the  weapon  system  is  in  the  acquisition  cycle, 
which  determines  the  availability  of  data  for  quantitative  measures. 

Functional  Design.  The  first  and  most  important  indicator  under 
functional  design  is  the  level  at  which  the  system  is  partitioned,  physically 
and  electronically.  The  optimum  system  from  a  maintenance  viewpoint  would  be 
100%  functionally  partitioned  at  the  LRU  level.  This  would  allow  a  failure  to 
be  identified  with  a  function  (receiver,  power  supply,  etc.)  and  the  entire 
function  to  be  replaced.  In  contrast,  in  a  fully  integrated  system,  all 
functions  are  interdependent.  When  one  fails,  the  possible  number  of  sources 
(i.e.,  the  suspect  set)  becomes  very  large.  The  resulting  problems  for  main¬ 
tenance  personnel  have  been  noted  in  various  studies  (3) . 

Many  new  systems  are  computer-controlled,  which  increases  inherent 
complexity.  Computer-based  systems  (e.g.,  P.^TRIOT,  DIVAD,  ROL.AND) ,  by  their 
very  nature,  are  integrated.  In  addition,  they  generally  require  a  signif¬ 
icant  amount  of  built-in  instructions  or  software.  The  more  software  (whether 
operating  program  or  self-test  program),  the  wider  the  range  of  possible 
suspect  sets  when  a  failure  occurs.  A  simple  indicator  of  this  added  com¬ 
plexity  is  the  number  of  lines  of  code  in  high  order  language  (HOL) . 

The  level  of  derating  of  components  also  affects  inherent  complex¬ 
ity.  The  actual  load  as  a  percent  of  design  capacity  not  only  has  a  thermo¬ 
dynamic  effect  on  the  failure  rates  of  the  components,  but  also  results  in 
failures  not  predicted  by  FMEA.  Such  anomalies  normally  manifest  themselves 
in  unpredictable  junction  breakdowns.  Thus,  a  system  with  components  derated 


TABLE  3-1.  INHERENT  COMPLEXITY  INDICATORS 


FUNCTIONAL  DESIGN 

1.  Partitioning:  percent  system  not  functionally  partitioned  by  LRU 

2.  Derating:  Percent  components  operating  above  60%  design  capacity 

3.  Tactical  Software:  lines  of  code  (HOL) 

4.  Maintenance  Diagnostics  Software:  lines  of  code  (HOL) 

5.  Number  of  disciplines  (electronics,  mechanical,  pneudraulics ,  optics) 

6.  Mix  of  technologies  (tubes,  discrete  solid  state,  IC,  LSI J 

FAILURE  MODES 

1 .  Number  of  failure  modes  identified  by  FMEA 

2.  Percent  DT/OT  II  failures  not  identified  by  FMEA 

3.  Percent  DT/OT  II  failures  identified  as  intermittent 

SELF-TEST  CHARACTERISTICS 

1.  Ambiguity: 

(a)  No  Indication:  percent  failures  processed  by  BITE  but  not  fault 
isolated 

(b)  False  Alarm:  percent  BITE  indications  reflecting  false  alarms 

(c)  Wrong  Indication:  percent  BITE  indications  that  are  wrong 

(d)  Ambiguity  Group:  complement  of  sensor  set  evaluation  coefficient 

2.  Interface  Requirements: 

(a)  Percent  identified  failures  requiring  manual  isolation  (machine- 
assisted) 

(b)  Percent  identified  failures  requiring  manual  interaction  (machine- 
directed) 

3.  BITE  Hardware  Reliability:  ratio  of  system  to  BITE  MTBF 
DIAGNOSTIC  REQUIREMENTS 

1.  Percent  failures  requiring  remove/replace  actions  for  fault  isolation 

2.  Percent  failures  requiring  peculiar  test  equipment  for  fault  isolation 

3.  Percent  failures  requiring  common  test  equipment  for  fault  isolation 

INTERCONNECTIVITY 

1.  Average  number  of  connections  among  individual  components 

2.  Average  number  of  connections  among  individual  subassemblies 

3.  Average  number  of  connections  among  assemblies 

SYSTEM  BREAKDOWN  STRUCTURE 


1.  Number  of  assemblies 

2.  Number  of  subassemblies 

3.  Number  of  components 

4.  Number  of  subsystems 

5.  Number  of  major  systems 


3-4 


to  a  high  percentage  of  design  capacity  (60%  -  75%)  will  be  inherently  more 
complex  than  the  same  system  with  the  same  components  derated  to  a  lower 
percentage  (say,  40%  or  below)  of  design  capacity. 


The  final  two  indicators,  disciplines  and  technologies,  only  affect 
inherent  complexity  if  the  weapon  system  involves  multiple  disciplines  and 
technologies.  For  example,  a  fire  control  system  controlling  a  mechanical  gun 
on  a  hydraulically  driven  turret,  all  controlled  by  a  computer  receiving 
signals  from  a  radar  set,  crosses  several  disciplines,  making  the  system 
inherently  complex.  The  suspect  set  associated  with  a  failure  may  be  in  one 
of  several  disciplines,  making  fault  isolation  difficult.  This  effect  would 
be  compounded  if  the  job  design  required  one  person  to  work  in  all  disciplines 
(see  section  on  job  complexity). 

A  similar  indicator  is  the  mix  of  technologies  utilized.  This 
refers  to  the  use  of  tubes,  solid  state  devices,  integrated  circuitry,  or 
large-scale  integration  at  the  component  level.  The  driving  factor  is  not  the 
use  of  a  particular  technology,  but  the  use  of  multiple  technologies  in  a 
single  system. 

Failure  Modes.  Failure  modes  identified  by  the  FMEA  are  indicative 
of  inherent  complexity.  The  FMEA  neve*"  identifies  all  failure  modes  complex 
equipment  will  exhibit  during  its  life,  however.  This  delta  is  commonly 
referred  to  as  failure  anomalies.  .Anamolies  have  a  serious  impact  on  trouble¬ 
shooting  because  no  procedures  have  been  developed  for  dealing  with  them,  nor 
are  their  symptoms  included  in  the  manuals.  The  number  of  anomalies  may 
increase  with  the  number  of  failure  modes  identified  in  an  FMEA,  and  this 
increase  may  be  non-linear.  Consequently,  the  number  of  failure  modes  identi¬ 
fied  by  the  FMEA  is  both  a  first-order  indicator  of  inherent  complexity,  and  a 
second-order  indicator  of  potential  troubleshooting  problems. 
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The  second  indicator,  percent  of  DT/OT  failures  not  identified  by 
the  FMEA,  indicates  the  "quality"  of  the  FMEA  performed  by  the  contractor,  and 
also  predicts  potential  troubleshooting  problems.  The  test  results  are 
normally  used  to  update  technical  manuals  and  job  aids,  but  may  also  indicate 
the  need  for  enhancements  of  BITE  hardware  and/or  software.  If  testing  time 
is  limited,  only  a  portion  of  all  possible  failure  modes  will  be  encountered 
in  the  testing  environment,  unless  a  sophisticated  failure  insertion  program 
is  followed.  Even  then,  simulated  failure  modes  will  normally  be  derived  from 
the  FMEA.  BITE  hardware  enhancements  are  notoriously  difficult  once  the 
detailed  hardware  design  has  been  completed.  For  all  these  reasons,  any 
DT/OT  II  failures  not  identified  by  the  FMEA  indicate  potential  maintenance 
problems . 

Some  of  the  failures  during  testing  may  be  classified  as  inter¬ 
mittent  failures,  detected  by  temporary  loss  of  an  operational  function  with 
or  without  a  corresponding  BITE  indication.  Systems  exhibiting  intermittent 
failures  are  inherently  more  complex  than  those  without;  their  diagnosis 
requires  a  high  level  of  proficiency.  Intermittent  failures  are  not  caused 
just  by  immature  hardware.  Fielded  complex  systems,  especially  those  with 
embedded  computers,  may  exhibit  this  phenomenon  too  (e.g.,  PERSHING  lA) . 
While  the  system  is  in  development,  intermittent  failures  will  normally  be 
minimized  through  redesign  efforts  based  on  such  techniques  as  "sneak  circuit 
analysis."  Apparently,  it  is  in  practice  infeasible  to  eliminate  all  inter¬ 
mittent  failures  in  a  complex  system.  The  consequences  in  terms  of  opera¬ 
tional  capabilities  and  maintenance  complexity  are  unpredictable. 

Self-Test  Characteristics.  The  first  four  indicators  listed  cor¬ 
respond  to  different  types  of  ambiguity  associated  with  the  self-test  or  BITE. 
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The  next  two  indicate  the  level  of  human  interface  required  to  fault  isolate 
using  BITE.  The  last  concerns  the  reliability  of  BITE  hardware. 


During  system  design,  requirements  for  automatic  fault  isolation  are 
determined  on  the  basis  of  reliability  predictions  and  maximum  allowable 
system  downtime.  Only  a  selected  set  of  failure  modes  normally  requires 
automatic  fault  isolation  to  bring  the  predicted  MTTR  down  to  an  acceptable 
level  (as  determined  by  system  availability  requirements).  The  level  of  BITE 
(specificity  of  fault  isolation  within  the  system  breakdown  structure)  is 
determined  on  the  basis  of  the  maintenance  concept  and  cost-effectiveness 
(.MTTR  reduction)  considerations. 

Implementation  of  BITE  may  result  in  four  types  of  ambiguity,  three 
of  which,  wrong  indications,  false  alarms,  and  ambiguity  group,  are  widely 
discussed  in  the  literature.  A  fourth,  percent  of  failures  not  resulting  in 
a  fault  indication,  is  not. 

Wrong  indications  result  from  automated  diagnostics'  incapacity  to 
distinguish  between  failure  modes  which  they  can  and  cannot  accurately  diag¬ 
nose.  Thus,  if  failures  occur  in  such  areas  as  interconnecting  wiring,  con¬ 
nectors,  cabling,  or  front  panel  components,  the  fault  indicator  will  normally 
be  erroneous  (27).  When  the  technician  replaces  the  module,  the  fault  indica¬ 
tion  remains. 

False  alarms  are  normally  divided  into  two  categories:  faults 
detected  during  system  operation  but  not  repeatable  during  the  fault  isolation 
process  (Type  1),  and  faults  detected  and  isolated  to  an  LRU  but  not  evidenced 
at  the  next  higher  maintenance  level  (Type  2).  Type  1  false  alarms  include, 
but  are  not  limited  to,  intermittent  failures.  If  the  indicator  used  includes 
these  failure  symptoms,  it  replaces  indicator  B.3  in  Table  3-1.  Type  2  false 
alarms  are  caused  by  testing  conditions  such  as  BITE  tolerances.  False  alarms 
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can  represent  a  significant  factor  in  assessing  maintenance  complexity;  Pliska 
(28)  shows  a  range  of  0.40  to  1.30  (Type  1)  and  0.25  to  0.65  (Type  2)  in  false 
alarms  (expressed  as  ratio  of  false  alarms  to  actual  failures)  for  a  repre¬ 
sentative  set  of  currently  fielded  electronics  systems. 

The  third  type  of  ambiguity,  commonly  referred  to  as  ambiguity 
group,  concerns  the  number  of  removable  items  included  in  the  set  to  which  the 
failure  is  isolated  by  BITE.  Some  failure  modes  may  be  cost-effectively 
isolated  to  a  single  LRU;  for  others,  the  BITE  may  not  isolate  beyond  a  siz¬ 
able  "suspect  set."  A  measure  commonly  used  for  this  BITE  resolution  level  is 
the  "sensor  set  evaluation  coefficient,"  defined  as  follows; 


where:  P.  =  percent  of  identified  failures  isolated  to  a  group  of  i 

^  LRUs 

P  =  percent  of  failures  identified  by  BITE 

( Note :  Greenman  (27)  defines  this  coefficient  in  terms  of  failure 
probabilities;  the  above  format  is  more  suitable  for  our 
purposes . ) 

The  higher  value  this  coefficient  has  (S  1.0),  the  higher  the  resolution  level 
of  the  BITE.  Thus,  the  corresponding  indicator  (C.l.d  in  Table  3-1)  is  meas¬ 
ured  in  terms  of  the  complement  of  this  coefficient  as  an  indicator  of  in¬ 
herent  complexity. 

A  fourth  type  of  ambiguity,  not  noticed  in  the  literature  but  po¬ 
tentially  the  most  critical,  is  the  percent  of  failures  not  resulting  in  a 
fault  indication  even  though  BITE  was  designed  to  detect  and  fault  isolate  the 
failure  modes  involved.  Of  course,  this  is  not  supposed  to  happen,  but  anec¬ 
dotal  evidence  seems  to  indicate  that  it  does.  Apart  from  failures  in  BITE 
hardware,  potential  causes  may  include  software  inadequacies. 
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The  next  two  indicators  relate  to  the  level  of  autonomy  at  which  the 
self-test  is  implemented.  Between  the  two  extremes  of  fully  manual  trouble¬ 
shooting  (no  self-test)  and  fully  automated  fault  isolation,  several  levels  of 
test  autonomy  may  be  distinguished.  Westinghouse  recommends  making  a  distinc¬ 
tion  between  two  intermediate  levels  of  test  autonomy:  machine-assisted, 
man-directed  and  machine-directed,  man-operated.  The  former,  also  referred  to 
as  lower-order  BITE,  would  provide  little  more  than  fault  detection,  with 
fault  isolation  left  to  the  technician.  The  self-test  would  not  fault  isolate 
below  two  levels  above  the  LRU  level  in  the  system  breakdown  structure  and 
would  give  no  further  troubleshooting  assistance.  The  latter  (higher-order 
BITE)  would  provide  machine  prompting  to  assist  in  troubleshooting.  Both 
levels  of  BITE  typically  might  require  manual  test  probing;  but  the  lower- 
order  BITE  would  not  direct  the  technician,  whereas  the  higher-order  BITE 
would . 

The  corresponding  two  indicators  in  Table  3-1  are  phrased  somewhat 
more  generically,  the  lower  level  connoting  manual  intervention  (requiring 
more  cognitive  skills),  and  the  higher  level  manual  interaction  (requiring 
fewer  cognitive  skills  but  the  ability  to  follow  instructions  precisely). 
Failures  in  either  cat  gory  increase  inherent  complexity. 

The  final  indicator  of  the  contribution  of  self-test  characteristics 
to  inherent  complexity  is  BITE  hardware  reliability  (MTBF) .  A  failure  in  BITE 
hardware  may  cause  various  ambiguities,  including  no  indication.  For  this 
reason,  BITE  hardware  reliability  should  be  an  order  of  magnitude  higher  than 
the  tactical  hardware.  The  ratio  of  the  two  is  an  indicator  of  inherent 
complexity. 

Diagnostic  Requirements.  All  failure  modes  not  addressed  by  fully 
automated  BITE  impose  a  diagnostic  burden  on  the  technician.  The  percent  of 
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failures  requiring  off-line  test  equipment  (common  test,  measurement  and  diag¬ 
nostic  equipment  (TMDE)  or  system-peculiar  special  test  equipment  (STE)) 
clearly  indicates  inherent  complexity.  Common  sense  would  suggest  that  fail¬ 
ures  requiring  STE  are  more  complex  to  troubleshoot  than  failures  requiring 
TMDE;  hence  Table  3-1  includes  an  indicator  for  each. 

The  proper  measure  for  these  two  indicators,  however,  should  reflect 
both  the  percentage  of  failures  involved  and  the  number  of  different  test 
equipments  required.  That  is,  a  nonlinear  scale  is  needed  to  permit  compari¬ 
son  of  the  diagnostic  requirements  for  two  systems;  for  example,  one  system 
with  10%  of  the  failures  requiring,  on  the  average,  5  pieces  of  test  equipment 
(the  remaining  90%  of  failures  representing  40%  of  the  failure  modes  are 
solved  by  fully  automated  BITE),  and  another  system  with  30%  of  the  failures 
requiring  2  pieces  of  test  equipment  (the  remaining  70%  of  the  failures  being 
solved  by  fully  automated  BITE). 

Another  diagnostic  technique  often  required  is  the  removal  of 
multiple  LRUs  (typically  circuit  boards)  to  isolate  the  failed  component. 
Procedures  needing  this  technique  may  be  viewed  as  increasing  the  complexity 
of  troubleshooting.  This  technique  is  typically  required  within  ambiguity 
groups  called  out  by  BITE  (see  previous  section  on  self-test  characteristics). 
It  is  often  necessary  in  fault  isolating  failure  modes  not  covered  by  BITE: 
complex  failures  (difficult  to  test)  with  a  very  low  frequency  are  seldom 
addressed  by  BITE  as  the  increased  cost  would  not  pay  off  in  reduction  of 
MTTR. 

In ter connectivity.  The  interconnectivity  of  weapon  system  parts  is 
a  basic  indicator  of  inherent  complexity.  The  higher  the  interconnectivity  at 
each  level  of  the  system  breakdown  structure,  the  larger  the  "suspect  set" 
associated  with  a  failure  symptom.  In  manual  troubleshooting,  perceptual  and 
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problem-solving  complexity  increase  with  interconnectivity;  in  self-test 


diagnostics,  software  complexity  (logic  and  lines  of  code)  increases 
similarly. 

IntercoEUiectivity  can  be  measured  in  different  ways.  Wohl  (2) 
applies  a  circuit  complexity  index  at  the  component  level,  defined  as  the 
average  number  of  components  connected  to  a  component.  As  stated  earlier,  his 
research  suggests  that  this  index  is  predictive  of  the  tail  of  the  repair  time 
distribution.  Rouse  (5)  demonstrates  that  two  different  measures,  "number  of 
relevant  relationships"  and  an  information  theoretic  measure  are  predictive  of 
fault  isolation  time.  (He  also  demonstrates  that  parts  count,  by  itself,  is 
not  a  good  predictor.) 

The  interconnectivity  indicators  in  Table  3-1  are  expressed  in  terms 
of  Wohl ' s  measure,  which  turns  out  to  be  closely  related  to  Rouse’s  number  of 
relevant  relationships.  Interconnectivity  at  the  lowest  three  levels  of  the 
system  breakdown  structure,  i.e.,  the  component  (smallest  removable  item), 
subassembly  (smallest  field  replaceable  unit,  e.g.,  PCB),  and  major  assembly 
(smallest  field  repairable  unit,  e.g.,  LRU)  levels,  may  be  most  predictive  of 
inherent  complexity. 

System  Breakdown  Structure.  Counts  of  the  weapon  system's  physical 
components  (printed  circuit  boards,  number  of  piece  parts,  number  of  cables, 
etc.)  provide  an  indicator  of  inherent  complexity.  For  example,  PERSHING  II 
is  being  touted  as  much  easier  to  maintain  than  PERSHING  lA.  This  expectation 
is  based  on  a  significant  reduction  in  the  number  of  "things’':  PERSHING  II 
has  53%  fewer  major  items  and  77%  fewer  cables  than  PERSHING  LA. 

The  indicators  listed  in  Table  3-1  are  expressed  in  a  generic 
terminology  for  the  indenture  levels  of  the  system  breakdown  structure.  The 
top  level  (first  indenture  below  weapon  system)  is  referred  to  as  the  major 
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system.  This  is  broken  down  into  subsystem,  major  assembly,  subassembly  and 
component.  For  a  specific  weapon  system,  the  actual  number  of  indenture 
levels  may  be  more  or  less.  The  maintenance  concept  determines  which  are  LRUs 
and  which  are  shop  replaceable  units  (SRlis).  The  relative  impact  the 
counts  by  indenture  level  (or  sequence  of  indicators  listed)  is  probably  a 
function  of  the  maintenance  concept.  The  sequence  given  in  Table  3-1  presumes 
that  assemblies  are  LRUs  and  subassemblies  SRUs . 

Comments 

The  above  indicators  clearly  overlap.  Specifically,  the  effects  of 
interconnectivity  (factor  E)  and  system  module  counts  (factor  F)  are  reflected 
in  the  number  of  failure  modes  (indicator  B.l)  and  ambiguity  measures  (indi¬ 
cator  C.l),  while  diagnostic  requirements  (factor  D)  may  have  little  to  add  to 
interface  requirements  (indicator  C.2).  Thus,  if  measures  for  the  first  three 
factors  were  available,  the  last  three  would  contribute  little  else  to  the 
overall  measure  of  inherent  complexity.  However,  no  measures  are  available 
for  factors  B  and  C  until  completion  of  full-scale  engineering  development. 
This  suggests  that  different  indicators  be  used  to  assess  the  maintenance 
complexity  of  new  systems  at  different  stages  of  the  development  cycle  (see 
Chapter  4) . 

A  second  point  which  may  need  clarification  is  the  absence  of 
certain  design  concepts  from  the  list  of  indicators.  For  example,  a  "soft 
fail"  design  (also  referred  to  as  "on-line  sparing")  permits  automatic  switch¬ 
ing  over  to  spare  circuitry  when  a  failure  occurs  in  a  specific  circuit.  This 
concept  permits  repair  of  the  failed  circuit  during  scheduled  maintenance. 
The  prime  effect  is  a  significant  increase  in  Aq.  Greenman  (27)  suggests  that 
soft-fail  design  also  facilitates  fault  isolation.  This  is  not  necessarily 


true;  the  concept  may  be  subject  to  the  same  ambiguity  problems  as  BITE. 


r 

Another  missing  concept  is  design  for  testability,  which  is  con¬ 
cerned  with  accessibility  of  identified  test  points,  and  a  host  of  other 
issues.  A  good  overview  of  testability  requirements  as  they  affect  mainte¬ 
nance  complexity  is  provided  by  a  draft  standard  for  testability  (16).  (Re¬ 
portedly,  this  reference  is  used  as  a  pilot  military  specification  for  test¬ 
ability  in  the  Army's  AAH  program).  The  net  effect  of  testability,  however, 
is  reflected  in  the  given  indicators  of  ambiguity. 

Another  such  design  characteristic  is  the  mix  of  analog,  digital, 
and  hybrid  modules.  These  parameters  are  often  used  in  MTTR  predictions  using 
linear  regression,  but  are  not  believed  to  affect  inherent  complexity  as 
such . 

With  regard  to  BITE,  Table  3-1  does  not  explicitly  include  as  an 
indicator  the  percentage  of  failures  fault  isolated  by  fully  automated  BITE 
(or  its  complement).  Instead,  the  effects  of  BITE  (apart  from  ambiguity)  are 
represented  in  terms  of  the  percentage  o.'  failures  fault  isolated  semi- 
automatically  (indicators  C.2).  It  may  be  of  interest  to  note  that  past 
attempts  to  use  regression  models  in  predicting  MTTR  have  shown  little  sensi¬ 
tivity  of  MTTR  to  the  percentage  of  failures  automatically  fault  isolated  by 
BITE  (28). 

JOB  (MOS)  COMPLEXITY 
Overall  Measure 

The  overall  measure  of  job  complexity  is  the  time  required  to  attain 
job  proficiency  at  the  journeyman  level,  as  measured  by  an  anchored,  benchmark 
scale  combining  learning  difficulty  and  aptitude/ability  prerequisites. 

Factors  and  Indicators 

The  factors  determining  job  complexity  are,  in  order  of  importance: 
cognitive  requirements 

1 
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task  diversity 
test  equipment  required 
information  load 
task  uniqueness 

Table  3-2  shows  the  indicators  associated  with  each  factor.  Some  indicators 
refer  to  the  number  or  percent  of  tasks,  which  presumes  that  a  complete  list 
of  the  tasks  allocated  to  the  job  (MOS)  is  available.  (The  list  should  in¬ 
clude  all  tasks  allocated  to  a  billet  in  the  logistic  support  analysis  record 
(LSAR)  or  QQPRI  even  if  that  billet  ultimately  is  not  included  in  the  draft 
plan  TOE;  i.e.,  tasks  that  may  be  overlooked  when  a  tasklist  is  prepared  from 
a  contractor-delivered  task  and  skill  analysis). 

TABLE  3-2.  JOB  (MOS)  COMPLEXITY  INDICATORS 

A.  COGNITIVE  REQUIREMENTS 

1.  Percent  tasks  with  incomplete  information  in  job  aids 

2.  Percent  tasks  requiring  technician  to  make  decisions 

3.  Percent  tasks  requiring  technician  to  interpret  information 

4.  Percent  tasks  requiring  pattern  recognition  ability 

B.  TASK  DIVERSITY 

1.  Number  of  different  end  items  supported 

2.  Number  of  area  specialties  required  (see  .Appendix) 

C.  TEST  EQUIPMENT 

1.  Number  of  system-peculiar  test  equipments  required 

2.  Number  of  common  test  equipments  required  in  job 

D.  INFORMATION  LOAD 

1.  Average  number  of  instructions  to  complete  a  task 

2.  Total  number  of  pages  of  TMs  and  other  job  aids 

E.  T.ASK  UNIQUENESS 

1,  Number  of  tasks  occurring  infrequently 

2.  Percent  of  infrequent  tasks  without  fully  procedurized  aid 
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Which  non-mission-essential  tasks  to  include  is  a  matter  of  judg¬ 
ment.  For  example,  at  a  direct-support-level  MOS,  maintenance  tasks  on  the 
special  test  equipment  used  to  check  out  LRUs  are  not  normally  coded  mission- 
essential.  However,  for  our  purposes,  such  tasks  should  be  included  in  the 
counts  driving  some  of  the  indicators.  A  logical  rule  would  be  to  include  in 
the  task  list  all  maintenance  tasks  for  all  end  items  the  MOS  is  responsible 
for,  and  to  exclude  such  nonactive  maintenance  tasks  as  supervision,  supply 
duties,  and  instruction. 

Many  MOSs  merge  at  the  higher  pay  grade  levels,  broadening  the 
variety  of  end  items  the  person  is  responsible  for,  primarily  as  supervisor. 
Thus,  "journejTnan  proficiency"  must  be  taken  generically:  the  intent  is  to 
limit  ourselves  to  active  maintenance  tasks  at  the  E-5  level,  but  to  include 
those  which  may  have  been  allocated  to  higher  pay  grades,  including  warrant 
officers  (e.g.,  troubleshooting  tasks).  The  Army  recognizes  five  skill  levels 
for  enlisted  personnel;  they  are  identified  as  skill  level  10  through  50  by 
the  last  two  digits  of  the  5-digit  MOS  code.  Skill  level  20,  which  is  the 
journeyman  level,  is  defined  as  pay  grade  E-5;  higher  skill  levels  essentially 
add  supervisory  and  instructional  duties  to  the  technical  skill  requirements 
of  the  MOS. 

A  brief  review  of  the  factors  and  their  indicators  follows. 

Cognitive  Requirements.  The  primary  factor  determining  job  complex¬ 
ity  is  the  level  of  cognitive  skills  required  for  successful  performance. 
Troubleshooting  tasks  are  more  complex  than  other  types  of  maintenance  tasks 
because  they  require  problem-solving,  perceptual,  and  information-processing 
abilities  in  addition  to  a  functional  understanding  of  the  system  and  its 
subsystems.  .A  job  (MOS)  specification  with  no  troubleshooting  tasks  implies  a 
simpler  maintenance  job  than  one  with  troubleshooting  tasks. 
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The  required  level  of  cognitive  skills  is  a  function  of  inherent 
complexity  (see  Table  3-1)  and  the  level  of  detail  and  accuracy  of  the  aids 
provided  to  the  technician,  be  they  built-in  software  diagnostics,  display- 
aided  maintenance  (off-line,  computer-controlled  prompting)  or  hard  copy 
manuals  (technical  manuals,  extension  training  materials,  skill  performance 
aids,  etc.)*  Even  if  a  specific  troubleshooting  task  is  fully  procedurized , 
it  normally  would  demand  some  cognitive  skills. 

A  typical  differentiation  of  cognitive  requirements,  as  shown  in 
Table  3-2,  consists  of;  decision  requirements,  incomplete  information  to 
determine  the  next  step  in  a  fault  isolation  procedure,  pattern  recognition 
requirements,  and  interpretation  requirements.  Given  a  job  task  list  and  the 
associated  aids  used  by  the  technician,  an  industrial  psychologist  would  be 
able  to  classify  the  cognitive  tasks  involved  in  terms  of  these  four  cate¬ 
gories.  An  appropriate  measure  for  these  indicators  should  take  into  account 
the  frequency  of  task  occurrences. 

Task  Diversity.  Task  diversity  is  defined  by  the  number  of  differ¬ 
ent  equipments  supported  by  the  same  MOS  and  by  the  different  area  specialties 
involved.  Both  indicators  should  probably  have  a  non-linear  scale.  The 
number  of  area  specialties  should  be  expressed  as  a  percentage  of  an  estab¬ 
lished  list  of  such  specialties  to  ensure  rating  consistency.  .A  preliminary 
list  of  electronics  specialties  is  shown  in  the  .Appendix. 

Test  Equipment.  Maintenance  personnel  may  have  difficulty  in  using 
test  equipment  proficiently.  The  amount  of  test  equipment  required  is  thus 
related  to  job  complexity.  It  is  more  difficult  to  acquire  the  necessary 
expertise  if  the  test  equipment  is  system-peculiar  rather  than  common.  Thus, 
Table  3-2  distinguishes  between  the  numbers  of  common  TMDE  and  system-peculiar 
STE  required  to  perform  the  job  tasks. 
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Information  Load.  Page  count  is  a  valid  indicator  of  job  com¬ 
plexity:  the  more  pages  in  a  manual,  the  greater  the  probability  of  inac¬ 
curacies  and  mistakes.  When  some  maintenance  MOSs  have  a  physical  count  of 
over  15,000  pages  of  manuals,  there  is  something  wrong  with  the  way  jobs  are 
designed.  Display  aid  maintenance  which  guides  the  technician  through  a 
procedure,  step  by  step,  can  simplify  a  task.  But  again  the  length  of  the 
procedure  (number  of  instructions  or  steps  required  to  complete  the  task)  is 
an  indicator  of  potential  for  errors,  both  on  the  part  of  the  technician  and 
the  aid;  and,  the  greater  the  potential  for  errors,  the  greater  is  the  time 
required  to  attain  job  proficiency. 

Task  Uniqueness.  Frequency  of  task  performance  is  a  basic  indicator 
of  a  technician's  proficiency.  With  the  increasing  reliability  of  electronics 
modules,  this  indicator  may  become  even  more  important.  Tasks  occurring 
infrequently  must  be  fully  procedurized . 

Defining  an  infrequent  task  is  difficult.  It  depends  on  the  pro¬ 
ficiency  decay  curve,  which  is  a  function  of  skill  level  and  the  discipline  or 
area  specialty  involved.  Little  research  has  been  found  on  this  subject  in 
the  literature.  The  ARTS  (12)  suggests  that  maintenance  proficiency  decays 
rapidly,  requiring  roughly  the  following  levels  of  on-the-job  reinforcement: 

remove/ replace  tasks,  mechanical  adjustments:  once  per  year 

mechanical  troubleshooting:  once  per  quarter 
electronic  troubleshooting:  more  frequently 
These  average  frequencies  apply  to  a  situation  where  tasks  are  carried  out 
under  strict  supervision;  without  supervision,  the  frequencies  required  to 
maintain  proficiency  would  be  higher.  We  suggest  that,  in  the  absence  of 
fielded  maintenance  training  simulators,  any  electronics  troubleshooting  tasks 
occurring  less  than  six  times  a  year  should  be  fully  procedurized. 
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Obviously,  there  is  a  relationship  between  proficiency  decay  and  the 
individual's  aptitude  and  cognitive  abilities,  but  we  have  not  located  any 
pertinent  studies.  Further  research  will  be  required  to  define  the  implica¬ 
tions  of  task  uniqueness  precisely. 

Comments 

The  maintenance  concept  affects  job  complexity  indirectly  through 
its  impact  on  the  indicators  just  described.  Defined  by  the  maintenance  al¬ 
location  chart,  the  maintenance  concept  indicates  what  tasks  are  to  be  per¬ 
formed  at  each  echelon  of  maintenance  (including  what  the  removable  items  are 
at  each  indenture  level).  The  complexity  of  these  tasks  is  affected  py  the 
system's  self-test  characteristics  (Table  3-1)  and  off-line  aids  such  as  JPAs 
(for  off-equipment  repairs  the  test  programs  available  on  automated  test 
equipment  affect  task  complexity).  The  allocation  of  these  tasks  to  specific 
MOSs  through  job  design  affects  job  complexity.  All  three,  m.iintenance  con¬ 
cept,  job  design,  and  aids  are  interrelated  and  should  be  part  of  the  same 
design  process.  Traditional  occupational  analysis  conducted  by  the  services 
using  standard  survey  questionnaires  and  the  Comprehension  Occupational  Data 
Analysis  Program  (CODAP)  is  not  specific  enough  to  permit  assessment  of  job 
complexity.  Foley  (39)  has  recommended  specific  formats  for  collecting  ad¬ 
ditional  data  required  to  better  integrate  the  front-end  analysis  process 
(task  and  skill  analysis,  instructional  system  development  and  JPAs). 

Job  complexity  can  be  controlled  by  assigning  the  most  complex  tasks 
such  as  troubleshooting  to  a  separate  cadre  of  experienced  technicians  outside 
the  enlisted  MOS  structure,  e  g.,  warrant  officers.  This  implies,  incident¬ 
ally,  that  assessment  of  maintenance  complexity  should  not  be  limited  to 
enlisted  personnel  requirements.  Chapter  2  has  commented  on  the  practical  and 
theoretical  limitations  of  JP.As  in  reducing  task  complexity;  specifically, 
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fully  procedurized  aids  for  digital  electronics  troubleshooting  have  so  far 
not  been  successful,  though  recent  research  shows  promise  (14). 


Finally,  whether  or  not  mission  criticality  should  influence  task 
and  job  complexity  is  a  matter  of  debate.  We  have  elected  to  avoid  this  issue 
by  proceeding  on  the  assumption  that  all  active  maintenance  tasks  specified 
for  the  job  are  equally  important. 

APPARENT  COMPLEXITY 
Overall  Measure 

Apparent  complexity  applies  to  a  specific  individual  in  a  sp  l  :  : :  c 
billet.  It  is  determined  by  the  gap  between  the  skills,  knowledge,  and  abil¬ 
ities  required  by  the  job  and  those  possessed  by  the  person.  If  selection, 
training,  and  personnel  utilization  were  perfect,  maintenance  complexity  would 
be  the  combined  outcome  of  inherent  and  job  complexity.  Apparent  complexity 
is  the  increase  in  job  complexity  caused  by  shortcomings  in  personnel  selec¬ 
tion,  training,  and  utilization.  The  proposed  measure  of  apparent  complexity 
is  the  percentage  of  job  tasks  the  technician  can  perform  correctly  within  set 
time  standards. 

Factors  and  Indicators 

The  problems  related  to  specifying  indicators  of  apparent  complexity 
were  discussed  in  Chapter  2.  Indicators  such  as  the  difference  between 
skills/  knowledge  required  and  training  standards,  and  between  basic  abilities 
required  and  possessed,  are  unknowable  without  meaningful  training  standards 
and  a  knowledge  of  basic  ability-job  relationships.  We  have  defined  a  number 
of  proxy  indicators  of  apparent  complexity  which  are  associated  with  the 
following  factors: 

-  person/billet  mismatch 
training  deficiencies 
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skill  utilization  deficiencies 
individual  differences 

Table  3-3  lists  these  factors  and  indicators.  A  brief  review  of  each  factor 
follows . 

TABLE  3-3.  APPARENT  COMPLEXITY  INDICATORS 

A.  PERSON/BILLET  MISMATCH 

1.  Duty  MOS  differs  from  primary  MOS 

2.  Incumbent  does  not  possess  ASI  required  by  billet 

3.  Incumbent's  skill  level  less  than  required  by  billet 

B.  TRAINING  DEFICIENCIES 

1.  Managed  OJT  deficiency 

2.  Unmanaged  OJT  deficiency 

3.  Apprentices  per  NCO  during  (M)OJT 

4.  School  practice  deficiency 

5.  Non-availability  of  training  equipment 

6.  Functional  theory  deficiency 

C.  SKILL  UTILIZATION 

1.  Diversions  from  MOS-related  work 

2.  Time  since  last  worked  on  equipment 

D.  INDIVIDUAL  DIFFERENCES 

1.  Reading  level  less  than  required 

2.  Aptitude  less  than  required 

3.  Psycho  motor  abilities  less  than  required 

4.  Cognitive  abilities  less  than  required 

5.  Number  of  times  recycled  in  technical  training 


Person/Billet  Mismatch.  Imbalances  between  job  requirements  and 
personnel  inventory  and  the  administrative  lead  time  required  for  assignment 
often  cause  mismatches  between  persons  and  billets.  None  of  the  Services 
exercises  centralized  control  over  assignments.  Units  periodically  receive 
replacements  matching  as  closely  as  possible  projected  vacancies  (=  authorized 
billets  minus  on-board  personnel)  in  terms  of  primary  MOS,  additional  skill 
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identifier  as  needed,  and  skill  level.  The  closeness  of  the  matches  is  sub¬ 
ject  to  the  logic  of  the  assignment  system,  imbalances  between  billet  author¬ 
izations  and  deployable  personnel,  and  unpredictable  events  occurring  between 
the  assignment  decision  and  the  assignee's  arrival.  The  actual  placement  of  a 
person  in  a  billet  is  the  commander's  prerogative. 

Mismatching  may  also  be  done  deliberately,  for  example  to  facilitate 
cross-training  and  the  acquisition  of  secondary  and  tertiary  MOSs.  .Mis 
matches  are  not  necessarily  bad,  but  they  do  affect  job  proficiency. 

Training  Deficiencies.  Lack  of  an  MOJT  program  is  a  prime  cause  of 
enlisted  personnel's  inadequate  maintenance  capabilities.  Informal  OJT  (as 
indicated  by  the  time  a  technician  spends  on  learning  tasks  spelled  out  in  his 
manuals)  might  compensate  for  this  to  some  extent,  if  enough  skilled  NCOs  were 
available  to  help  the  apprentice. 

To  quantify  the  contribution  of  MOJT  and  OJT  shortcomings  to  ap¬ 
parent  complexity,  their  requirements  must  be  known.  Unfortunately,  a  train¬ 
ing  plan  developed  under  the  instructional  system  development  approach  does 
not  normally  spell  out  these  requirements  beyond  identifying  the  critical 
tasks  not  taught  in  school  and  developing  the  corresponding  extension  training 
material  for  use  in  the  field.  The  .Army  does  not  have  a  training  cadre  for 
conducting  OJT;  there  is  no  MOJT  program  for  any  of  the  electronics  mainte¬ 
nance  MOSs  we  have  studied;  and  it  is  doubtful  in  any  case  whether  novices  can 
learn  complex  tasks  solely  through  serving  as  helpers  on  the  job.  Pending 
further  research,  any  attempts  to  quantify  (M)OJT  deficiencies  would  have  to 
be  based  on  expert  judgment. 

Potential  causes  of  school  training  (AIT)  deficiencies  include; 
lack  of  hands-on  training  (equipment  is  often  down  for  maintenance  or  supply); 
lack  of  the  theory  required  to  understand  cause-effect  relations  within  a 
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system;  more  emphasis  on  rote  learning  than  on  cognitive  skills;  and  devia¬ 
tions  from  the  program  of  instruction  (POl).  These  indicators  could  be 
quantified  in  terms  of  actual  versus  required  time,  assuming  that  the  POI 
defined  the  requirements.  However,  since  most  of  the  entry-level  courses  are 
self-paced,  the  material  completed  will  be  in  accordance  with  the  given  POI. 
The  question  then  becomes  whether  the  POI  reflects  the  "true"  training  re¬ 
quirement.  In  the  absence  of  indicators  predictive  of  POI  quality,  this 
question  can  only  be  answered  through  analysis  of  observed  values  of  the 
criterion  measure,  actual  job  proficiency.  However,  Mallory  and  Elliot  (13) 
show  the  potential  of  using  a  hardware-free  test  for  assessing  a  student's 
troubleshooting  skills. 

Skill  Utilization.  In  the  field,  MOS  diversions  and  improper  use  of 
personnel  in  response  to  operational  requirements  may  detract  from  job  pro¬ 
ficiency.  For  example,  field  data  suggest  that  an  Army  maintenance  technician 
(skill  level  20  and  below)  typically  may  spend  30%  of  available  time  on  non- 
MOS  related  duties  (1).  This  obviously  limits  the  opportunity  to  gain  tech¬ 
nical  proficiency.  Or  there  may  be  such  specialization  within  an  .MOS  that  the 
technician  loses  proficiency  in  other  tasks  related  to  that  MOS.  For  most 
MOSs  responsible  for  a  variety  of  end  items,  the  items  at  the  duty  station 
determine  what  equipment  the  technician  will  be  working  on.  Upon  reassignment 
to  another  duty  station  where  different  end  items  are  fielded,  that  technician 
may  be  less  proficient  than  the  skill  level  would  suggest. 

Individual  Differences.  Of  the  indicators  listed  under  "Individual 
Differences"  in  Table  3-3,  only  reading  level  is  both  measurable  and  valid  in 
terms  of  apparent  complexity.  If  an  individual's  reading  level  is  below  that 
for  which  the  job  manual  was  written,  the  apparent  complexity  of  the  job  will 
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be  affected.  The  other  indicators  either  are  available  but  have  poor 
validity,  or  have  demonstrated  validity  but  are  currently  unavailable. 

The  only  measures  of  aptitude  now  available  are  obtained  from  the 
ASVAB.  The  electronics  aptitude  composite  (EL)  is  the  single  selection 
standard  for  all  Army  electronics  maintenance  MOSs ,  with  cutting  scores  rang¬ 
ing  from  90  to  110  depending  on  MOS  (the  Army  uses  a  standardized  scale  of  40 
to  160  with  aptitude  scores  of  100  representing  the  mean) .  The  EL  currently 
combines  the  following  four  subtests  with  equal  weights:  general  sciences, 
arithmetic  reasoning,  mathematics  knowledge  and  electronics  information;  thus, 
it  is  both  an  aptitude  and  a  knowledge  test.  However,  as  previously  discussed 
in  Chapter  2,  the  EL  does  not  predict  job  performance  (37).  (Nor  does  the 
reading  level  which  correlates  well  (r  =  0.8)  with  the  ASVAB  composite  tests 
(38).)  The  need  for  better  selection  test  batteries  is  well  recognized  (34). 

Each  MOS  has  an  EL  prerequisite  which  is  seldom  waived,  but  its 
validity  is  suspect  (see  previous  discussion  of  job  complexity).  Thus, 
neither  aptitude  as  measured  by  ASVAB,  nor  deviation  from  stated  prerequisites 
is  necessarily  indicative  of  apparent  complexity. 

Basic  task-performance-related  abilities  are  currently  not  recog¬ 
nized  nor  tested  for  in  the  Army,  even  though  research  has  shown  their 
validity  (9,  32). 

Data  on  recycling  in  AIT  are  difficult  to  obtain.  Recycling  may  be 
caused  by  aptitude  and  motivation  problems  predictive  of  future  job  perform¬ 
ance,  but  may  also  be  caused  by  basic  learning  problems  which  could  be  over¬ 
come  by  teaching  students  how  to  learn  (15). 

Comment 

.Apparent  complexity  receives  little  attention  in  weapon  system 
acquisition.  Its  consideration  during  the  design  process  is  limited  to 
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"target  audience"  descriptors  normally  provided  by  the  user  community  to  the 
system  developer.  .Apparent  complexity  is  not  a  primary  responsibility  of  the 
program  manager,  but  of  the  TRADOC  and  the  user  commands. 

.Apparent  complexity  must  be  monitored  to  ensure  that  the  planned 
support  system  is  not  defeated  through  poor  implementation.  Consider,  for 
example,  what  has  happened  to  the  support  of  the  common  FM  radio,  the  AN/VRC- 
12  family.  The  design  of  this  radio  was  based  on  modularization,  with  com¬ 
ponents  replaceable  in  the  field.  MOS  31V  (radio  mechanic)  was  to  do  most  of 
the  repairs  at  the  organizational  level,  while  MOS  31E  (radio  repairer)  would 
check  out  and  repair  the  LRUs  at  the  direct  support/general  support  levels. 
The  failure  of  this  rational  maintenance  concept  over  the  past  decade  would 
take  longer  to  document  than  space  permits.  Briefly,  the  basic  causes  were  as 
follows : 

The  test  equipment  for  the  31V  did  not  perform  as  planned. 

The  TMs  included  incorrect  procedures  for  troubleshooting. 

The  training  of  the  31V  was  deficient. 

As  a  result,  the  standard  operating  procedure  both  in  USAREUR  and 
FORSCOM  for  the  past  several  years  has  been  to  by-pass  the  organizational 
level,  and  to  evacuate  the  faulty  radios  to  direct  support  (in  FORSCOM  units 
most  radios  end  up  at  the  civilian  Director  of  Industrial  Operations  support 
organization  at  the  post).  While  corrective  actions  are  being  implemented, 
recent  surveys  have  demonstrated  that  most  of  the  radios  were  in  non-operative 
condition  and  probably  had  been  so  for  many  years.  What  should  have  been  a 
very  simple  maintenance  job  (low  scores  on  inherent  complexity  and  job  com¬ 
plexity)  became  a  nightmare  in  terms  of  apparent  complexity  due  to  inadequate 
operational  testing  and  monitoring  of  job  performance. 
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Quantitative  measures  for  most  indicators  of  apparent  complexity  are 
simply  not  available.  Instead  of  attempting  to  predict  apparent  complexity, 
efforts  should  be  focused  on  further  improving  its  measurement  by  means  of  job 
task  performance  testing. 
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4.  VALIDATION  AND  APPLICATIONS  Of  TAXONOMY 


INTRODUCTION 

The  preceding  two  chapters  have  described  a  methodology  for  assessing  the 
maintenance  complexity  of  weapon  systems.  Maintenance  complexity  consists  of 
three  major  components;  inherent  complexity,  comprising  factors  which  are 
purely  equipment-related;  job  complexity,  comprising  factors  associated  with 
the  maintenance  tasks  allocated  to  each  MOS;  and  appa’^ent  complexity,  compris¬ 
ing  factors  representing  the  gap  between  the  capabilities  required  by  the  job 
and  those  possessed  by  the  person  assigned  to  it.  A  criterion  measure  was 
defined  for  each  of  the  three  complexity  components.  For  each  of  tne  con¬ 
tributing  factors,  a  set  of  indicators  was  identified,  permitting  assessment 
of  the  contribution  of  each  factor  to  the  first  two  of  the  criterion  measures. 

Much  more  needs  to  be  done  to  convert  the  taxonomy  into  a  useful  tool  for 
assessing  maintenance  complexity.  First,  research  is  required  to  develop  the 
criterion  measure  of  job  complexity;  an  anchored,  benchmark  scale  combining 
expert  ratings  of  learning  difficulty  and  aptitude/ability  prerequisites  for 
each  maintenance  job  in  the  Army.  Second,  institutionalized  job  proficiency 
measures  such  as  job  task  performance  tests  (e.g.  the  Army's  SQT)  must  be 
validated  before  they  can  be  used  as  criterion  measures  of  apparent  com¬ 
plexity.  Third,  the  given  indicators  of  inherent  and  job  complexity,  their 
metrics,  and  their  relationships  to  the  two  associated  criterion  measures  must 
be  tested  and  validated. 

Once  this  work  has  been  accomplished,  the  taxonomy  could  be  applied  as  a 
standard,  analytic  tool  to  predict,  identify  or  measure  logistic  support 
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deficiencies  caused  by  the  inability  of  maintenance  personnel  to  perform  their 
maintenance  tasks  as  planned. 

VALIDATION  APPROACH 

Validation  of  the  proposed  taxonomy  would  be  a  major  undertaking,  re¬ 
quiring  full  cooperation  and  participation  from  the  Service  involved.  The 
•Army,  faced  with  the  largest  jump  in  hardware  complexity  through  the  force 
modernization  planned  for  the  1980s,  would  benefit  most  from  a  serious  analy¬ 
sis  of  maintenance  complexity. 

Validation  would  entail  the  following  efforts.  First,  the  job  complexity 
scale  must  be  developed  with  ratings  of  difficulty  to  learn  and  aptitudes 
required  for  a  sample  of  maintenance  jobs.  The  Army  Research  Institute  would 
be  the  logical  organization  to  lead  this  effort,  which  could  build  upon  the 
related  work  performed  by  the  Air  Force  (10).  Second,  quantitative  data  must 
be  collected  on  all  indicators  of  inherent  and  job  complexity  (Tables  3-1  and 
3-2)  for  the  same  sample  of  MOSs  and  associated  weapon  systems.  This  effort 
would  require  participation  by  DARCOM  and  TRADOC.  Third,  the  sample  data  must 
be  subjected  to  statistical  analysis  to  determine  the  relationships  between 
criterion  measures  and  indicators.  The  appropriate  scales  or  metrics  for  the 
indicators  must  also  be  determined.  Fourth,  additional  sample  data  on  other 
.MOSs  and  weapons  systems  must  be  obtained,  repeating  the  first  two  steps. 
Using  the  equations  determined  earlier,  predicted  values  of  the  criterion 
measures  must  be  compared  with  the  actual  ones  for  this  second  sample.  Due  to 
the  relative  "softness"  of  some  of  the  data,  the  process  may  have  to  be  iter- 
■ited  until  an  acceptable  confidence  level  has  been  obtained. 

It  may  be  advisable  to  start  off  with  a  few  pilot  applications  of  the 
taxonomy,  prior  to  a  full-scale  validation  program,  to  ascertain  whether  the 
taxonomy  makes  sense  and  is  capable  of  explaining  variances  in  maintenance 
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capabilities.  In  view  of  the  limited  sample  data  available  in  such  a  pilot 
effort,  statistical  analysis  may  need  to  be  supplemented  with  expert  judgment. 
One  could  convene  a  select  group  of  senior  NCOs  or  warrant  officers  with  the 
requisite  technical  experience,  supplemented  with  representatives  of  the 
schools  where  the  MOSs  involved  are  taught,  and  MILPERCEN.  Using  a  DELPHI- 
type  of  approach,  average  values  of  the  relative  weights  of  the  indicators 
could  be  obtained  from  the  participants'  assessments.  Such  an  expert  panel 
would  also  be  helpful  in  reviewing  the  taxonomy  and  determining  the  appropri¬ 
ate  scales  for  the  indicators.  On  the  other  hand,  using  technical  experts  to 
directly  quantify  job  complexity,  skill  level  and  training  requirements  based 
on  an  early  engineering  data  package  (without  the  aid  of  a  taxonomy  as  pro¬ 
posed  here)  results  in  inconsistent,  invalid  estimates  (31). 

.APPLICATIONS 

Potential  applications  of  the  taxonomy  are  twofold:  (1)  micro  analysis 
of  the  maintenance  complexity  of  a  new  weapon  system  as  part  of  the  acquisi¬ 
tion  review  process,  and  (2)  macro  analysis  of  maintenance  complexity  for  MOS, 
mission  area  or  Service  aggregates.  Each  application  focuses  on  different 
aspects  of  the  taxonomy.  Both  types  of  analysis  are  comparative.  Macro 
analysis  identifies  trends  in  maintenance  complexity  over  time;  micro  analysis 
compares  the  maintenance  complexity  for  a  new  system  with  that  of  the  system 
it  will  replace  or  with  that  of  another,  comparable  system  in  the  inventory. 

Micro  Analysis 

Figure  4-1  illustrates  the  potential  use  of  the  taxonomy  in  the 
weapon  system  acquisition  process.  The  bottom  portion  of  the  figure  identi¬ 
fies  the  factors  on  which  the  review  process  should  focus--factors  limited  to 
inherent  and  job  complexity. 
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FIGURE  4-1.  MICRO  ANALYSIS  OF  MAINTENANCE  COMPLEXITY 


Macro  Analysis 

Figure  4-2  illustrates  a  potential  application  of  the  taxonomy  in 
assessing  trends  in  job  complexity  within  a  selected  set  of  MOSs .  The  curve 
depicting  trends  in  apparent  complexity  as  measured  in  the  past,  in  combina¬ 
tion  with  estimated  future  changes  in  job  complexity,  may  suggest  needed 
improvements  in  personnel  selection,  training,  or  support  concepts. 
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APPENDIX 


EXAMPLE  OF  AREA  SPECIALTY  LIST 


Task  diversity  is  one  factor  influencing  the  job  (MOS)  complexity  index. 
One  indicator  of  task  diversity  is  the  number  of  different  area  specialties 
which  the  MOS  must  master  (Table  3-2).  To  use  the  taxonomy,  it  would  be 
desirable  to  have  a  complete  list  of  area  specialties  within  each  discipline 
(electronics,  mechanical,  pneudraulics ,  optics,  etc.).  Compilation  of  such  a 
list  is  probably  best  left  to  subject  area  experts.  An  example  of  an  area 
specialty  list  is  shown  on  the  following  pages.  It  lists  the  subject  areas 
identified  in  the  Air  Force's  Electronic  Principles  Inventory  (EPI)  program 
which  is  used  in  their  occupational  survey  work  (4) .  The  EPI  program  recog¬ 
nizes  1,257  "electronics  principle  items"  classified  into  62  "subject  areas" 
and  21  "duty  groups." 
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TABLE  A-1.  EXAMPLE  OF  ELECTRONICS  AREA  SPECIALTIES 


Duty 

Number 

Group 

Area  Specialty 

of  EPI 

1. 

Mathematics 

14 

A 

2. 

Direct  Current  and  Voltage 

9 

3. 

Resistance 

28 

1. 

Multimeter  Uses 

9 

B 

2. 

Alternating  Current 

6 

3. 

Inductors  and  Inductive  Reactance 

25 

1. 

Capacitors  and  Capacitive  Reactance 

36 

C 

2. 

Transformers 

43 

3. 

Magnetism 

14 

1. 

RCL  Circuits 

44 

D 

2. 

Series  and  Parallel  Resonance  (Time  Constants) 

10 

3. 

Filters 

22 

1. 

Coupling 

12 

E 

2. 

Soldering 

22 

3. 

Relays 

19 

1. 

Microphones 

13 

F 

2. 

Speakers 

15 

3. 

Oscilloscopes 

12 

1. 

Semiconductor  Diodes 

50 

G 

2. 

Transistors 

24 

3. 

Transistor  Amplifiers 

49 

1. 

Solid-State  Special  Purpose  Devices 

6 

H 

2. 

Power  Supplies 

29 

3. 

Oscillators 

0  -7 

«i.  i 

1. 

Multivibrators 

16 

I 

2. 

Limiters  and  Clampers 

10 

3. 

Electron  Tubes 

44 

1. 

Electron  Tube  Amplifiers  and  Circuits 

7 

J 

2. 

Special  Purpose  Electron  Tubes 

16 

3. 

Heterodyning,  Modulation,  and  Demodulation 

6 

1. 

AM  Systems 

28 

K 

2. 

FM  Systems 

19 

3. 

Numbering  Systems 

10 

1. 

Logic  Functions 

13 

L 

2. 

Boolean  Equations 

25 

3. 

Counters 

24 
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TABLE  A- 

1.  EXAMPLE  OF  ELECTRONICS  AREA  SPECIALTIES 

(Continued) 

Duty 

Number 

Group 

Area  Specialty 

of  EPI 

1. 

Timing  Circuits 

12 

M 

2. 

Use  of  Signal  Generators 

10 

3. 

Motors  and  Generators 

29 

1. 

Meter  Movements 

10 

N 

2. 

Saturable  Reactors  and  Magnetic  Amplifiers 

16 

3. 

Waveshaping  Circuits 

11 

1. 

Single  Sideband  Systems 

30 

0 

2. 

Pulse  Modulation  Systems 

39 

3. 

Antennas 

39 

1. 

Transmission  Lines 

31 

P 

2. 

Waveguides  and  Cavity  Resonators 

50 

3. 

Microwave  Amplifiers  and  Oscillators 

76 

1. 

Registers 

7 

Q 

2. 

Storage  Devices 

9 

3. 

Digital  to  Analog  Converters 

14 

1. 

Phantastrons 

1 

R 

2 . 

Schmitt  Triggers 

3 

3. 

Cable  Fabrication 

2 

1. 

Input/Output  Devices 

3 

S 

2. 

Photo  Sensitive  Devices 

1 

3. 

Synchronous  Vibrations  (Chopper  Circuits) 

9 

1. 

Infrared 

27 

T 

2. 

Lasers 

34 

3. 

Display  Tubes 

14 

U 

1. 

Programming 

21 

2. 

DB  and  Power  Ratios 

3 
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